5 TITLE OF INVENTION 

FLEXIBLE EFFICIENT BRANCHING PARTICLE TRACKING ALGORITHMS 

10 Be it known that we, Michael A. Kouritzin, a citizen of Canada, and 

Klaus E. Fleischmann, a citizen of Germany, have invented new and useful 
improvements in the above-mentioned invention, of which the following is a 
specification. 

15 BACKGROUND OF THE INVENTION 

The present invention relates to a method and a system for efficient non- 
n linear optimal tracking, smoothing, and prediction of dynamic processes in real 
%B time 

2Q Many industries require a system to determine the location and state of a 

«| signal in some domain. For example, one might want to know the position, 
H orientation, and velocity of an aircraft over a busy airport. Additionally, it might 
7 be important to know where the aircraft is likely to be in the future and what 
path it has taken in the past. In many such cases full information about the 
2fS signal is not available. Instead, only distorted, corrupted, partial information 
p given by discrete-time observations is provided. In the aircraft example, the 
?== only data provided to the airport might be noisy transmissions of truncated 
altimeter and range data from the aircraft that arrive at two second intervals. 
While perfect determination of the signal state is impossible under these 
30 circumstances, it is still desirable to obtain probabilistic estimates of the state 
conditioned on the information that is available. Preferably one should also 
obtain smoothed estimates of past signal states, probabilistic information on 
the likelihood of various signal paths, and predictions of future states given all 
observations up to some time. 
35 More precisely stated, the problem is as follows. The signal X t to be 

tracked is a cadlag Markov process, for example it may follow an Ito equation 

dX t = A(X t ) + B{Xt)dB t , 



5 or more, generally a Martingale problem on a complete separable metric space. 
The discrete observations Y tkl occurring at times t fc , ^ e dependent on the signal 
in a stochastic manner described by 

Yt h = h k (X th ,V k ), 

where V k is a sequence of random variables that model the noise input into this 
10 time-varying dependence. These formulae might not be precisely known in a 

particular instance and some extra randomness can be introduced to model 

this implementation uncertainty. 

The requirement is to construct a device which can efficiently determine 

the conditional distribution of the state of the signal given all observations up 
15 to the current time, that is, 

- | P{X th Zdx\Y u ,i<k). 

fn 

Sj This device is known as an optimal tracker. Also, the device should 

f?i provide smoothers to estimate past signal states and predictors to estimate 
2dH future signal states, that is, , 

s and .. .. 

[a where r a <t k and ft > Indeed, one is often interested in 
2 -P(^]€^|^,i<A0, 
25 where z ranges over the cadlag functions and X[ r .,r,\ ■ represents the 

path of the signal between times , T a and r p . 

In the specific Ito equation case in which the stochastic dynamics given 
by A and B are Jinear, the observation function, h is of the form 

h k{X th , V k ) si h k xXt k .+ V k (additive noise), 
30 and each . V k is an independently distributed Gaussian random variable, an 
efficient recursive optimal tracker is already known: the Kalman filter. Re- 
cursive, in this context, means that the computation involved with each new 
observation requires the data of that observation only with no direct reference 
to the data of any previous observation. This allows the Kalman filter to operate 
35 very quickly. However, as noted, the Kalman filter has very stringent 
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5 requirements on the possible signal dynamics and on the form of the 
observation noise. As well, it provides no direct method to provide predictors or 
smoothers. 

Despite these shortcomings, the Kalman filter is the core of most current 
devices to provide filtering solutions. In linear Gaussian additive environments, 
10 trackers based on the Kalman filter produce optimal estimates. For more 
general environments the Kalman filter is suboptimal, but can still provide 
some solution in the form of the extended Kalman filter. Later developments 
have included running multiple Kalman filters in banks, each with different 
assumptions about the signal dynamics, and weighting the Kalman filters and 
15 their inputs to provide a track. This method is called the interacting multiple 

model tracker, and provides improved but still suboptimal tracking. 
CO New techniques have expanded the class of filtering problems that can be 

^rj solved using exact methods. These techniques provide readily implementable 
computer algorithms that are nearly as efficient as the Kalman filter and 
20 provide optimal solutions. However, the problems that can be solved in. this 
p manner still form a small subset of the whole space of real world problems. 
f! A more recent type of general solution to the filtering problem is the set 

Ul of particle methods. These techniques simulate a large number of independent 
u samples $ of the signal, called particles. The set of particles ff, 0 < j < N are 
25 then adjusted in some manner to conform to each observation, possibly by 
assigning each a scalar weight value W$ or by interchanging the state values 
associated with each particle in some manner. The conditional distribution of 
the state of the signal at the current time is approximated by the locations of 
the particles and their weights, specifically, 

2L/j=i ^tjb i=i 

The particle adjustment is performed in such a manner that the resulting 
weighted particle distribution converges to the optimal conditional distribution 
of the signal, given the observations, as the number of particles, N, is 
35 increased. Predicted distributions for some time r p > t k conditioned on ob- 
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5 servations up to time tfc are obtained by simulating the particles forward in 
time without any adjustment. 

Two possible particle methods are described in U. S. Patent 5,933,352 
published on Aug. 3, 1999 in the name of Gerard Salut, entitled "Method and a 
system for non-linear optimal estimation of dynamic processes in real time". In 
10 the first system, the claimed method of adjusting the particles to match the 
observations is solely to maintain a weight value associated with each particle 
and to modify this weight in accordance with each observation. This method 
has two sub-methods, in one of which the influence of past observations on the 
particle weights is attenuated in time, and in the other of which the influence 
llS of past observation on the weights is strictly limited in time. 

A second system claimed by U. S. Patent 5,933,352 is to adjust the 
particles after each observation by redistributing some or all of their state data, 
flj This random redistribution is conducted such that particles with a higher 
f« s weight are more likely to have their state data overwrite the state data of other 
2Q particles that are part of the redistribution process. In this manner, the particle 
Hi states are preferentially collected around the areas of the state space which 
j s § have the highest conditional probability of containing the signal. While U.S. 
p Patent 5,933,352 does not specifically state this, it can be assumed that the 
intention is for all particles that are involved in this redistribution to have their 
25 weights set to the average of the weights of all involved particles after the 
redistribution is complete, since this is the condition which will retain the 
asymptotic optimality of the system. 

As U. S. Patent 5,933,352 states, simple particle weighting without redis- 
tribution causes degeneration in the output conditional distribution that can 
30 be rectified only to some degree by attenuating or limiting the effect of older 
observations. These actions themselves entail an unavoidable loss of asymp- 
totic optimality in increasing numbers of particles for the filter. Tf only some 
particles are redistributed, there are still some degenerative effects working 
against optimality, and the calculation of the average weight for a subset of the 
35 particles is cumbersome. 
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5 However, a great amount of computation is involved in redistributing all 

of the particle states at each time step. This effort is especially pronounced if 
the particle data must travel between processing units in a distributed 
architecture. Moreover, redistributing all of the particles instead of merely 
considering each particle to determine whether redistribution or branching (as 

10 discussed below) is appropriate for it induces undue extra randomness that 
impairs path space capabilities and downgrades performance. 

Neither of the systems claimed in U. S. Patent 5,933,352 provide 
smoothing distributions, that is, estimates of the signal state for times r a < t k , 
given the observations up to time . They also provide no estimates of the 

15 historical path of the signal. 

|j BRIEF SUMMARY OF THE INVENTION 

pf The present invention relates to a method and a system for efficient non- 

jU linear optimal tracking, smoothing, and prediction of dynamic processes in real 
26 time. The method used is to store a large number of independent samples of 
!L. the signal state, known as particles. The branching particle-based nonlinear 
CH filter accepts a sequence of measurements from sensors, each of which contain 

: : 

[if noisy, corrupted, and distorted information about the position of the signal (i.e. 

y "measurement noise"), adjusts its particles in a novel manner such that they 
25 conform to the measurements,, and then uses these particles to provide 
smoothing, tracking, and predicting approximate conditional distributions of 
the signal state. These distributions can be used to provide a human display 
interface or to control automated systems. 

30 BRIEF DESCRIPTION OF THE DRAWINGS 

The method of the invention is illustrated by the drawings in which: 

Figure 1 shows the filtering process; 

Figure 2 shows the filter observation process; 

Figure 3 shows the evolve particle process; 
35 Figure 4 shows the branch particle process; 
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5 Figure 5 shows the renormalize particle count process; 

Figure 6 shows the duplicate particle process, and 
Figure 7 shows the remove particle process. 

DETAILED DESCRIPTION OF THE INVENTION 
10 .In accordance with the method of the invention, a filter is shown in 

Figure 1. In processing each observation (Figure 2: Filter Observation), the filter 

i 

first evolves each particle state forward in time (Figure 3: Evolve Particle) and 
removes (Figure 7: Remove Particle) particles that have left the bounds of the 
signal domain, if the particle evolution allows this. Each particle is then 
15 branched (Figure 4: Branch Particle) according to the observation, and after 
>=) each particle has been processed, the total particle count is renormalized 
Co (Figure 5: Renormalize Particle Count). 

J To evolve a particle (Figure 3: Evolve Particle), the particle state is first 

j y copied to a separate logical location in the device, and the original is marked as 
2©| the direct ancestor of the new particle. A forward evolution of the state of the 
L new particle is then simulated according to the stochastic law of the signal 
p 1 process, or if that is not available then some perturbed approximate model of 
L| the signal process. The time period for this forward evolution is given by the 
length of time between the preceding observation and the observation presently 
25 under consideration. " . . 

After evolving each particle, the particles are then branched (Figure 4: 
Branch Particle) to account for the information- from the observation. A value 
labeled is calculated for a given particle x in response to the observation 
at time h according to tiie formula 
30 • tk(x)=rmn{\C k (;x)ll}'Sgn{C k (x)}, 
where 

c * ( } " w L 

35 in this formula, 6 k is the density function for the random variable ' V k 

that models the noise input, the function. gk (Y k} x ) = V k is the inverse map of 



6 



the function h k (x,V k ) = Y k that describes the observations, and MY kl x) is the 
determinant of the matrix {^}(Y ki x)}^ =l . These formulae are constructed 
specifically so that the "branching method will provably converge to the optimal 
filter as the number of particles is increased. 

For additive zero mean, cr variance Gaussian noise such as the 
Kalman filter requires, the formula for Cfc becomes 

Similarly to incompletely understood signal dynamics, the model of the 
noise which is used to form 9 k and 9k(Y k , x) \ can be approximate if no better 
knowledge is available. '. 

A uniform-(0,l) random variable U is generated for the particle to be 
branched, and a comparison is .then made between this U and (k(x)- U >= ! 

.16 (a) I then the particle is not affected by the branching. This is common, 
especially (because of the form of the calculation of £ k (x)) with more difficult 
types of noise that -roos treduce the information contained in an. observation. " 

If U < |& (x) | then, in the case that &(z) > 0, ■ the particle is duplicated 
(Figure 6: Duplicate Particle), and in the case that < 0, the particle is 
removed (Figure 7: Remove Particle). 

To re-normalize the total, particle count (Figure 5: Re-normalize Particle 
Count), a value l is set to equal the difference between the current number of 
particles and the original number. If the current number of particles is greater 
than the original number, then I of those particles are selected at random to be 
removed (Figure 7: Remove Particle). If the current number of particles is less 
than the original number, then if this number is. less than half of the original 
number of particles, all particles are duplicated (Figure 6: Duplicate Particle), 
updating the value of Teach time, until the current number of particles is 
.within half of the original number, and then I of the particles are selected. at 
random to be duplicated again. At the end of either possibility the current total 
number of particles is equal to the original number. 



Duplicating a particle (Figure 6: Duplicate Particle) is performed by first 
copying the particle state to a separate logical location in the device. The count 
of the number of particles is then incremented, the new particle is marked as 
having the same direct ancestor as the original, and the weight value of all 
ancestor particles (chaining back recursively to include the marked ancestors 
of ancestor particles) is incremented by one. 

To remove a particle (Figure 7: Remove Particle), first delete the particle 
state data and decrement the count of the number of particles. Then decrement 
the weight of all ancestors (chaining back recursively as described above) of the 
deleted particle, and if the weight of any of these ancestors is reduced to zero, 
then delete that ancestor particle. 

As is the case for any particle filter, the particle locations provide the 
information to construct the approximated conditional distribution of the signal 
state. For the optimal tracking filter, the current particles are used with weight 
value of one for each. To construct an optimal predicting filter, evolve a copy of 
the current particles forward to the time for which the prediction is to occur, 
and use a weight value of one for each. 

This new branching particle method also allows the construction of 
optimal smoothing niters. The purpose of the ancestor particles is to retain 
probabilistic data about, the likely historical path of the signal. To use these 
particles to form a smoothing filter for time Ta < t k) , select the set of ancestor 
particles that were created at time t k „ where <= r a but is the most recent 
observation time for which this is true. Copy each of these ancestor particles 
and evolve them forward to the time r s , if r a ^ t ka , Then these particles, 
weighted by their associated ancestor particle weights, provide the approximate 
conditional distribution of the signal state at time +»• If it is known in advance 
for which times smoothing filters will be required, then the state of each 
particle at that time in its evolution can be stored as ancestor particles as well, 
and no forward evolution will be required during the construction of a 
smoothing filter. The weight for each of these intermediate ancestor particles 
will be the same as that of their direct ancestor. Ancestor particle data 
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5 older than a certain amount of time can be discarded in order to bound the 
resource usage without disrupting estimates within the remaining period. 

The branching particle filter operates recursively on the observation data, 
allowing real-time operation of the system. It is asymptotically optimal in 
increasing numbers of particles and in a decreasing period of time between 

10 observations, but the rate of convergence with regards to the observation 
period is extremely fast, so that the difference between the approximation and 
the optimal filter for practical observation periods is essentially dependent on 
the number of particles. The particle data can easily be distributed among 
multiple hardware systems such that operations are efficiently computed in 

15 parallel, allowing large numbers of particles to be used practically. In a 

r»i distributed system, re-normalization of the particle count can be conducted 
among each of the hardware units separately, or a more complicated scheme 

Sj including particle transfer among hardware units can be implemented if data 

r[i transfer within the system is rapid. 

20 A major improvement of our branching particle filter over other methods 

s is the cautious branching, by which is meant the high probability that particle 
In branching will not affect a particle rather than duplicating or removing it. This 
f = ~; aspect of the method allows a considerable increase in the speed of the filter, 
□ which can be traded for an increase in the number of particles within the same 
25 amount of processing power, and thus a closer approximation to the optimal 
filter. Even at the same particle count as other particle filters, the branching 
particle filter evinces superior performance in high observation noise cases 
since there is a smaller variance induced by particle re-distributions. This 
means that there is a smaller probability that the particles will be redistributed 
30 to erroneous locations. For example, the particles may be less likely to 
redistribute themselves to a single or a few selected previous particle sites in a 
too focused manner and erroneously attribute too great a commitment to an 
incorrect determination of the signal state. These advantages are more 
apparent as the observations become more rapid or as the observation noise 
35 becomes more extensive, both of which are situations that increase the 
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practical difficulty of the filtering problem and for which good solutions are 
most useful, since in these cases the particles are even less likely to be affected 
during a branch. In situations of extreme observation rate and noise distortion, 
it is possible with some noise models to approximate and shorten the 
calculation of the ~fc value without disrupting the asymptotic optimality of the 
filter, and thereby to decrease the calculation time for each particle further. 

The cautious branching is not merely an alternate, faster, and more 
accurate method for approximating the optimal filter; it is a device which allows 
the storage of information required to provide optimal smoothers and full 
historical path filtering by ensuring that past state information is properly 
associated with each particle. The method of U. S. Patent 5,933,352 and other 
particle filters destroys this information by redistributing particles to one of a 
randomly selected set of states at each observation period, disregarding the 
path that the particle took in the state space to arrive at this new location. The 
availability of this full historical path data allows the system to provide an even 
larger set of information than just optimal smoothers. For example, the particle 
system can be queried to determine the probability that the signal moved from 
one location to another through a particular path at a particular time. This is a 
major advancement in the capabilities of practical, general purpose nonlinear 
filters. 
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